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Abstract 

In our global society with its increasing 
international competition and tighter financial 
resources, governments, commercial entities and 
other organizations are becoming critically aware of 
the need to ensure that space missions can be 
achieved on time and within budget. This has 
become particularly true for the National 
Aeronautics and Space Administration’s (NASA) 
Office of Space Science (OSS) which has 
developed their Discovery and Explorer programs 
to meet this need. As technologies advance, space 
missions are becoming smaller and more capable 
than their predecessors. The ability to predict the 
mission success of these small satellite missions is 
critical to the continued achievement of NASA 
science mission objectives. The NASA Office of 
Space Science, in cooperation with the NASA 
Langley Research Center, has implemented a 
process to predict the likely success of missions 
proposed to its Discovery and Explorer Programs. 
This process is becoming the basis for predicting 
mission success in many other NASA programs as 
well. This paper describes the process, 
methodology, tools and synthesis techniques used 
to predict mission success for this class of mission. 

Nomenclature 

AO Announcement of Opportunity 

CDR Critical Design Review 

CoDR Concept Design Review 

EMI Electromagnetic Interference 

GDS Ground Data System 

NASA National Aeronautics and Space 

Administration 

OSS Office of Space Science 

PDR Preliminary Design Review 

PRR Preship Readiness Review 

WBS Work Breakdown Structure 


Background 

In response to the constrained budgets and the 
science communities need for more frequent 
scientific flight investigations, the Office of Space 
Science has developed an acquisition strategy 
designed to reduce the overall cost and schedule of 
small satellite missions. This reduction has been 
achieved by soliciting entire science investigations 
from the scientific community, capitalizing on the 
strengths of open competition and peer review. 

The science investigator is responsible for the total 
mission, including the development of the science 
objectives, the design of the instruments, spacecraft, 
and mission operations centers and the collection of 
science data and the subsequent data analysis. A 
process for evaluating these science investigations 
from a technical, management and cost perspective 
has been developed by OSS and NASA Langley 
Research Center to allow the best achievable 
science to be conducted. 

To ensure equivalent science investigations can 
compete on an equal footing, the Explorer Program 
is divided into three different classes of missions: 
Medium Class Explorers (MIDEX), Small Class 
Explorers (SMEX), and University Class Explorers 
(UNEX). OSS solicits proposals for each of these 
about every two years. The Discovery Program is 
not divided into different programs and solicits 
proposals approximately every 18 months. The 
Discovery Program and MIDEX solicitations are 
usually conducted through a two-step process with 
formal proposals submitted in response to an 
Announcement of Opportunity followed by a 
competitive funded study period with 4-6 proposals 
selected from the first step. The Small Explorer and 
University Explorer investigations are usually 
selected for flight after only a single proposal stage. 
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When acquiring science investigations through this 
technique, the Office of Space Science typically 
assesses science investigations proposed to 
Discovery and Explorer Program series with 4 
evaluation criteria: 

1. Scientific merit of the investigation 

2. Feasibility of achieving that science objectives 

3. Feasibility of the mission implementation 

approach 

4. Social benefits of the investigation 

Typically, two separate panels conduct the 
evaluations. A science peer review panel is 
responsible for assessing the scientific merit of the 
science investigation and the feasibility of 
achieving the science objectives. A technical, 
management, cost and other program factors 
(TMCO) panel assesses the feasibility of the 
mission implementation approach and the social 
benefits of the investigation. For proposals that 
respond to the second step of the two-step 
approach, OSS usually convenes a single panel to 
assess the results of the funded study. In general, 
this second review examines the higher fidelity 
designs and development plans, but does so in a 
similar fashion to the initial evaluation. Unless the 
science has changed, the scientific merit of the 
investigation is not reassessed. Once selected for 
flight, the mission is subjected to at least one more 
formal confirmation review. Again, this review 
will look at the same elements of the mission, but at 
the level of fidelity at the Preliminary Design 
Review (PDR). This review is consistent with 
NASA's review requirements specified in the 
NASA Procedures and Guidelines (NPG) 7 120.5 A, 
Program and Project Management, dated April 3, 
1998. 

Evaluation Overview 

The ability to predict the successful achievement of 
space science missions proposed to the Discovery 
and Explorer programs requires examination of all 
elements of these proposed science investigations. 
These facets include: scientific objectives; scientific 
instrument capabilities and development; mission 
design; spacecraft design, development and 
integration; launch integration and operations; 
mission and science operations; and finally science 


data analysis and archival. Each of the 
implementation aspects is examined by scientific 
peers and technical and management experts to 
determine the likelihood that the element can 
support the scientific objectives. The results of 
these analyses are synthesized and integrated into 
cohesive conclusions. Since the uncertainties 
driving a team’s ability to deliver the hardware and 
software necessary to achieve scientific success 
become clearer as the science investigation/mission 
proceeds through its design and development, the 
ability to predict improves as the development 
progresses. To compensate for uncertainties early 
in development, the technical, cost and schedule 
resources are examined carefully to ensure that 
there are adequate reserves and margins for the 
current stage of development. 

• The technical approach is examined in detail to 
determine the level of understanding and 
feasibility of the technical design details, the 
areas of unknowns, the trades necessary to 
resolve those unknowns, and the technical 
resources (for example, mass, power, data), 
particularly the reserves and margins that are 
available to handle the uncertainties and risks; 

• The management and organization are 
examined to determine if the project 
organization matches the development 
approach and provides adequate skills, 
processes and management tools to ensure 
problems can be foreseen as they arise; and 

• The costs and schedules are analyzed to ensure 
that there are adequate resources to meet the 
engineering, manufacturing, integration, 
operation and data analysis elements of the 
project required to achieve the scientific 
objectives. 

Since the success of the technical development and 
subsequent mission and science operations depends 
on both the management and organization and the 
cost and schedule resources, ah analyses are 
synthesized and integrated through a 
comprehensive process of examining the 
interrelationships between the three different areas. 
This synthesis is the essence of NASA’s ability to 
predict mission success. Each of the three areas 
(technical, management and cost/schedule) will be 
described in detail, followed by a description of 
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how these three areas are integrated to arrive at an 
overall prediction of mission success. 

Predicting Technical Success 

Most space mission teams proceed through a 
similar and standard design and development 
process. Once the team has established it’s 
scientific objectives (or other objectives such as 
commercial communication) and the payload 
necessary to meet these objectives, they begin to 
look at the overall requirements that the payload 
will place on the rest of the system architecture. 
From this the team can develop a mission concept 
and a system architecture which satisfies the basic 
system requirements. This typically includes the 
weight, power and data requirements for the 
payload. From this the team can estimate the 
weight, size, power, data and communication 
requirements for the spacecraft and ground systems. 
Given the weight and size of the spacecraft the team 
can determine the appropriate launch vehicle and 
mission design to meet the technical objectives. 
Following the development of the system 
architecture, the team refines the design through 
increasing technical definition, system trade studies, 
and subsystem design. This process usually follows 
a similar path from the concept development and 
technical definition, through preliminary design, 
detailed design, fabrication, assembly, integration 
and test leading up to delivery of the system to the 
launch site. At each stage of design and 
development the unknowns are reduced and the 
margins for uncertainty are better defined. If teams 
are using the current state of the art, there is no 
reason why they shouldn’t be able to deliver the 
planned system (with all technical objectives met) 
on schedule and within budget. If this is true, then 
why are some missions successful and others not. 
Although technical difficulties do not in themselves 
lead to failure, there are generally three reasons that 
contribute to why missions are not successful. 

First, mission teams assume that they understand 
their mission well enough to ignore the standard 
rules of thumb on design margins such as mass 
margins. Second, teams assume that they will not 
encounter significant difficulties in developing new 
technologies or advancing the state of the art 
necessary to enable their mission. Third, teams do 
not methodically, rigorously, and judiciously 
control and safeguard project resources (margins for 


mass, power, cost, schedule etc.) throughout the 
project life cycle. As the realities of system design 
and development occur, teams are forced into using 
up margins and taking greater risks to overcome 
development problems. 

The OSS/LaRC technical evaluation process is a 
systematic approach to examining each element of 
the system design, development approach, and 
operations plans. The evaluation teams include 
experts in management, mission design, spacecraft 
design and development, instrument design and 
development, mission operations, and ground data 
system design and development. Since OSS 
examines missions at various stages of 
development, these evaluation teams expect the 
level of design/development maturity to be 
commensurate with the point of the planned overall 
development schedule. 

Mission Design . The evaluation begins with 
examining the overall mission architecture to ensure 
that the mission team has considered all the 
elements of the system completely and correctly. 

In early stages of concept development, it is not 
uncommon for teams to develop architectures 
which have elements that do not work well 
together. The criticality of this mistake depends on 
the degree of freedom that the team has left 
themselves in terms of technical, schedule and cost 
margins to solve problems. If teams have adequate 
margins, this type of mistake may not be considered 
a showstopper, but developing workable 
architectures will ensure that the mission is viewed 
in a positive light. The system architecture is also 
examined from a complexity and flexibility 
standpoint. How hard is the mission to accomplish 
correctly and what degree of flexibility does the 
team have if problems occur during the mission? 
After examining the system architecture, the 
technical parameters of the mission design (launch 
energy and trajectory, orbit parameters, deep space 
trajectories, etc.) are verified through orbital 
mechanics analytical tools. In the early stages of 
development these analytical tools require 
definition of assumptions (e.g., propulsion Isp) 
which can affect the outcome. It helps when the 
teams provide sufficient information to minimize 
the assumptions made by an evaluation team. This 
is particularly critical when discussions between the 
proposing team and the evaluation team are limited. 
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This case usually occurs during the formal 
solicitation process. For OSS missions the follow- 
on reviews usually allow adequate discussion to 
illuminate misunderstandings. However, in 
attempts to maximize the return on investment in 
terms of the maximum payload, many teams make 
overly optimistic assumptions which may erode 
mass and power margins later in the development. 
Finally, mission duration and launch window 
constraints are considered when assessing the 
likelihood that the mission can sustain long 
operations periods or be launched during tight 
launch windows. 

Spacecraft Design. The level of maturity of the 
spacecraft design is evident by the depth of the 
system and subsystem descriptions, block diagrams 
and equipment lists. Experts familiar with 
spacecraft design determine if the various sub- 
systems interface appropriately, are composed of 
appropriate hardware and software and have 
adequate contingencies for the heritage and 
complexity of the design and the development stage 
(See figures below. Note: these contingency 
represent minimum reserves for expected growths. 
Margins for unplanned/ unexpected growths must be 
added on top of these contingencies). In addition, 
the level of redundancy and reliability is examined 
in light of the length and risk of the mission. When 
new technology or advances in the current state-of- 
the-technology are required, back-up alternatives to 
these technologies are reviewed to ensure that they 
are feasible and can be implemented at the proper 
time. If no back-ups are planned, the schedule and 
cost margins are particularly important as the 
methods for managing the technology risks. 

The mission duration and the design life of the 
spacecraft are carefully considered when examining 
the levels of redundancy, reliability and parts 
quality planned for the spacecraft. The ability of 
the spacecraft to gracefully degrade can alleviate 
some reliability concerns, but the level of spacecraft 
reliability needs to be commensurate with the 
mission duration and impact of failure to 
accomplish the mission objectives. 


Although the trend is toward integrated hardware 
and software development, software development is 
sometimes overlooked by proposers as a critical 
development area. Review teams expect to see a 
rational sequence and approach for developing, 
testing and verifying the flight hardware and 
software. 

During the last 10 years, the size and cost of 
missions have come down considerably. This has 
allowed more missions to be accomplished. Since 
this has permitted the simpler missions to be 
accomplished, new missions are beginning to be 
significantly more complicated. The number and 
complexity of the mechanisms are risk factors that 
must be handled appropriately. In these cases, the 
degree of hardware qualification for the planned 
environment and operation is particularly important. 
Accommodating failsafe features into these 
complex systems enhances the mission's reliability 
by providing opportunities to continue other 
portions of the mission when failures occur. If 
failsafe features are not incorporated, missions may 
not be successful. 

In the early stages of the design process, the 
mission team is faced with many system trades in 
arriving at the most optimum design. The plan for 
and the status of these trades, the conclusions drawn 
and the processes used to conduct these trades 
reveal how well the team is addressing the many 
design decisions. 

To substantiate expert opinions and to ensure 
thoroughness in our reviews, analytical design tools 
are used to verify that the proposed spacecraft has 
the resources necessary to meet system 
requirements during mission operations. Although 
these tools are not perfect, they highlight design 
areas where resource stresses occur and allow 
reviewers opportunities to examine, in depth, how 
well the design operates. Thus, mission teams will 
benefit from including the best and most complete 
information in proposals or review packages. 


4 



Minimum Standard Weight Contingencies (Percen 

Ref. 1 



- Production Spacecraft - 
Mass <500kg 

■ Next Generation spacecraft 

- Mass <500kg 
New Spacecraft - Mass 
<500kg 

Production Spacecraft - 
Mass <50kg 

- Next Generation spacecraft 

- Mass <50kg 

■ New Spacecraft - Mass 
<50kg 


Minimum Standard Power Contingencies (Percent: 

Ref. 1 



■Production Spacecraft - 
Power < 1500 watts 
-Next Generation spacecraft 

- Power < 1500 watts 
New Spacecraft - Power < 
1500 watts 

Production Spacecraft - 
Power < 500 watts 
■Next Generation spacecraft 

- Power < 500 watts 

- New Spacecraft - Power < 
500 watts 
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Payload Design ( including Payload to Spacecraf t 
interfaces). The payload is examined in much the 
same way as the spacecraft. The maturity and 
completeness of the designs are compared to the 
postulated point in the development process to 
ensure that planned development activities are 
appropriate. Margins, technologies and 
redundancies are evaluated in light of the payload 
complexity, technology readiness and heritage. The 
complexity of the payload is compared to the state 
of development and qualification plans. Finally, the 
payload requirements on the spacecraft and ground 
systems are compared with the level of technical 
resources (e.g., power) available to ensure that the 
payload will be allowed to perform as specified. 

The payload-to-spacecraft interfaces are examined 
for complexity and thoroughness. A traceability 
matrix, which starts with the science objectives and 
walks through the mission, instrument, spacecraft 
and ground system requirements is extremely 
helpful in understanding how well the system is 
designed to meet the mission objectives. 

Mission Operations and Ground Data Systems 
( GPS ) Design. The mission operations and ground 
data system designs must be compatible with the 
spacecraft and payload designs so the mission 
planning and subsequent operations plans are 
evaluated to ensure they reflect a reasonable and 
adequate approach. The mission operations and 
ground data system architecture can be as important 
as the flight hardware and software and can have as 
big an impact on mission success. Missions have a 
better chance of succeeding when the mission 
operations and ground data systems are developed 
concurrently with the flight system. This allows 
both systems hardware and software to begin 
working together early enough to identify problems 
that might occur. Mars Pathfinder used this 
technique to reduce the cost of the GDS by several 
factors by using existing hardware and software as 
part of the flight system test equipment, and most of 
the Discovery missions are following this example. 

The technical aspects of the mission design affect 
how well the mission can be operated and thus must 
be carefully examined against the operations 
concept. The spacecraft operations can be complex 
requiring very sophisticated software that must be 
verified prior to use. If the software is not planned 
to be loaded on the spacecraft prior to launch, then 


a version of the spacecraft hardware and software is 
required on the ground. Having this type of 
capability has also proven very effective when 
trying to resolve spacecraft anomalies after launch. 
The communication links between the spacecraft 
and the ground stations must have sufficient margin 
to ensure that the data can be adequately retrieved. 
The geometry between the spacecraft, the Earth and 
the Sun can affect this greatly. Since the ground 
antennas can be heavily committed, the need for 
these assets or the need to build new ones must be 
planned. When existing facilities are a significant 
part of the mission operations strategy, it is 
important to demonstrate the availability of these 
assets through commitments by the asset owners. 

Finally, the drive for lower cost mission operations 
is leading to more innovative approaches. These 
new approaches must be carefully thought out and 
tested before fully implementing. CONTOUR 
(Comet Nucleus Tour), Discovery’s sixth mission, 
is designed to go into a hibernation mode between 
comet encounters. During this period, the 
spacecraft is place in a safe mode and left 
unattended for months at a time. Given the recent 
Lewis spacecraft failure, where the spacecraft was 
left unattended for brief periods; i.e. ; weekends 
(and subsequently lost), this method will need to be 
carefully proven prior to full implementation. 

Development Approach ( including manufacturing, 
integration and test). 

The development approach is considered as 
important to mission success as the ability of the 
design to meet its requirements. Having an 
adequate approach and sufficient time to 
accomplish the development is critical to success. 
Many mission teams are going to a protoflight 
development approach (where the development unit 
is ultimately flown). Although this increases risk 
somewhat, it is proving acceptable in missions 
where technology is not a big driver. When new 
technology is required, more traditional 
development strategies with prototypes, 
brassboards, breadboards, engineering models, etc. 
allow development problems to be rectified prior to 
investing in the flight hardware. Regardless of the 
development strategy chosen, each element of the 
development plan is carefully looked at to ensure 
that implementation of the design can be achieved. 
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The schedule is compared to the planned 
development activities to ensure that sufficient time 
is available to meet delivery requirements and to 
overcome development problems that will occur. 
Depending on the degree of difficulty of the 
development, one month of schedule slack per year 
of development time is a reasonable rule of thumb. 
If the development is particularly difficult, at least 
1.5 months of slack/year may be necessary. 

The methods for hardware flight qualification are 
an important aspect of the development plans. If 
heritage hardware is planned, the team needs to 
consider how close the qualification of the original 
hardware matches the planned usage. Many teams 
are now routinely performing heritage reviews to 
identify how well the hardware meets the planned 
system performance. Part of the Lewis failure, 
however, has been attributed to inappropriate use of 
heritage (hardware use did not match the original 
usage) which resulted in a flawed attitude control 
system design. For new pieces of hardware, 
particularly mechanisms, lifetime testing is also 
needed to enhance the likelihood of successful 
operation in orbit. The Genesis spacecraft 
(Discovery’s fifth mission), which has extensive 
mechanisms, is planning life testing at over 2 
lifetimes to ensure confidence that the hardware 
will work on orbit. 

As always, software development and testing must 
be matched to the hardware development. 
Concurrent software development allows the 
hardware and software to be tested in testbeds early 
in the development flow so that the bugs can be 
worked out. When software development is 
decoupled from the hardware flow, additional 
schedule reserve needs to be included to work out 
the software bugs. Early testing also allows for 
more burn-in of electronic components. Adequate 
burn-in is often overlooked or considered a luxury. 
Increasing the burn-in can increase confidence that 
the early failure modes have been eliminated, 
particularly for single string systems. 

The completeness of environmental testing to verify 
that the system will operate as designed in the 
various mission environments (radiation, 
temperature, solar, vibration, acoustic, EMI, etc.), is 
also examined. Skipping tests when schedule or 
cost pressures arise increases the risk to mission 


success. Equally important is having adequate time 
in the schedule to resolve flight system anomalies 
found during testing. Invariably, workmanship or 
design flaws are found during this period that must 
be resolved prior to launch, and if at all possible, 
retested. 

Finally, the launch site activities are examined to 
ensure that the team has allowed adequate time for 
shipping, flight system to launch vehicle 
integration, hardware testing, fueling, etc. For 
many missions, the launch window is very short, 
therefore it is critical that launch preparations have 
adequate schedule time with margin to accomplish 
these tasks. 

When examining the development approach, the 
review team also looks carefully at the various 
development plans and processes to ensure that they 
are appropriate and complete for the given stage of 
development. Adequate systems engineering is an 
essential part of both the design and the 
development process. Through this discipline, 
integration and orchestration of the entire 
design/development is accomplished, and 
resolutions to development problems can be worked 
without inadvertently impacting other parts of the 
system. In addition, system engineering maintains 
track of system resources and configurations. Test 
and verification plans reveal the degree to which the 
mission team plans to qualify, test and verify 
system performance. Adequate configuration 
control gives confidence that what is being tested 
matches plans. Finally, the quality assurance 
program is examined to assure that the plans and 
processes will ensure an acceptable quality product. 
ISO 9000 certification is now becoming a 
requirement and the degree of certification 
improves confidence that the mission team has the 
right processes in place. 

Predicting Management Success 

The successful execution of the design and 
development processes are dependent on an 
effective management approach. The 
organizational structure needs to be matched to the 
work and should be as simple as possible. 

Simplicity allows less complicated project control 
techniques and processes, but is not always 
possible. Many projects have large organizations 
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with each organization responsible for an element 
of hardware or software. As the complexity of the 
organization grows, the importance of effective 
project control techniques and tools is increased. 
Appropriate performance measurement systems and 
receivable/deliverable systems go along way to 
keeping management informed of development 
progress. Coupled with these, a good systems 
engineering team and systems engineering process 
will ensure that decision data is the best information 
available. In both large and small organizations 
responsibility, authority and accountability need to 
be matched and the decision-making process clear 
to all members of the team. Many development 
problems can be attributed to confused lines of 
authority, which inhibit decisions necessary to 
move through the development process. A key 
indicator of the quality of the management 
approach is how well the work breakdown structure 
corresponds to the organizational structure. When 
these are poorly matched, the lines of authority and 
responsibility are obscured. 

Despite the desire by organizations to want to move 
into new business areas, successful past experience 
in performing the functions assigned is a very good 
indicator of the likelihood of success in similar 
functions for new missions. Unfortunately, 
organizations without experience, do not usually 
fully understand all the pitfalls related to a given 
task. The quality and experience of key personnel 
is equally important. Most of the new smaller 
mission are fast tracked with little margin for error. 
Individuals in leadership positions must have 
adequate training and experience if the project is 
expected to be delivered on time and within budget. 
The project personnel can expect to work overtime 
so full time or near full time commitments are 
important to ensure adequate attention to details. 

As obvious as this may seem, the continuing 
pressures to do more with less encourages 
organization to assign more responsibilities than is 
sometimes prudent. 

Procurement strategies and subcontract 
management cannot be overlooked since much of 
the mission hardware will be procured. Mars 
Pathfinder assigned a procurement manager to track 
the progress of procurements, and this proved 
extremely beneficial. This person assisted the 
project manager in deciding the best procurement 


strategy, including incentive plans, and reported on 
the progress of hardware development and 
deliveries. This allowed early warning of 
procurement problems so that adequate 
management attention could be applied to mitigate 
delivery issues and to develop work around plans. 

Regardless of the management approach, risk 
management and mitigation is critical to the success 
of any project. Without adequate risk management, 
it is impossible to predict whether a given mission 
can be successful, and NASA has now made this 
mandatory for all NASA projects. A 
comprehensive and tailored risk management plan 
should be developed as soon as possible in the 
project life cycle, and then followed rigorously until 
the project is complete. This plan needs to 
encompass not only technical issues, but schedule 
and cost as well. Risk management and mitigation 
involves understanding the project's entire 
technical, cost and schedule envelope (all margins) 
so that problems may be resolved through the 
application of the appropriate resource. For 
example, mass problems might be solved through 
the application of cost reserves, or technology 
development problems might be overcome with 
schedule reserves. Review teams look for evidence 
that the mission team understands the risks 
associated with their individual mission and have 
planned actions to deal with them. The 
identification of appropriate specific risk items, the 
affects of those risks on the project and the methods 
for analysis, tracking and mitigation (e.g., fallback 
options) indicate a good understanding of the 
problems ahead and how to keep them constrained. 
Descope plans are also a necessary part of the risk 
management plan, but these should be used as a last 
resort. When descope plans are developed, it is 
very important to identify for each descope item the 
appropriate decision date and the impact on the 
mission. 

Predicting Cost Success 

Predicting cost success is one of the most 
misunderstood elements of predicting mission 
success. Most of this can be attributed to a lack of 
understanding on how good cost analysis is 
conducted. This lack of understanding comes in 
part from the decoupling of cost from the technical 
aspects of the mission and from a lack of proper 
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utilization of analytical cost tools. However, cost 
modeling is no different than technical analytical 
modeling. The results are only as good as the 
correlation of the analytical model with the system 
being measured. Unfortunately, correlation of cost 
modeling is not as easy as it may seem. The 
OSS/LaRC cost analysis method is designed to 
provide the most complete assessment possible. 

The figure below is a graphic representation of the 
elements of good cost analysis and how each 
element builds to form a comprehensive view of a 
given mission’s budget. The objective of this cost 
analysis is to verify the mission team’s cost 
estimates, not to develop a NASA estimate for what 
the mission should cost. As always, it behooves the 
proposer to provide as much and as convincing data 
as possible to substantiate their proposed costs. 
However, many teams assume too much optimism 
in the development process without supporting 
evidence. This usually leads to a higher risk 
assessment conclusion by the evaluation team. 

The cost analysis begins with a detailed review of 
the project’s cost documentation including: the 
WBS, basis of estimate (rationale used by the team 
in developing their estimate), funding profile, 
project schedule, staffing plan, costs by 
organization, contributions (elements provided at no 
cost) and rationale for savings (heritage, new ways 


of doing business, etc.). This review is designed to 
examine the consistency of all the cost pieces and 
the rationale behind them. Once the review of the 
team’s data is completed, the technical data from 
the mission are input into two different cost models. 
These models require a number of assumptions (e.g. 
heritage and technology readiness levels), and these 
assumptions are derived from the technical data 
presented by the mission team. The results of the 
two cost models are compared, and the differences 
reconciled. These results are then compared with 
the mission team’s estimate. When differences are 
observed, the differences are examined by cost and 
technical members of the review team to determine 
whether the cause is poor correlation or if the 
mission team’s estimate appears unrealistic. 
However, the cost model results are never used 
exclusively. Historical cost actuals for similar 
items are compared with both the cost model results 
and with the mission team’s estimate. This data 
provides an independent credibility check of 
projected cost estimates. Based on data provide by 
the technical and management teams and data from 
the mission team, cost threats are identified and 
quantified, if possible. The analytical cost analysis 
is combined with the cost threats, risk items and 
risk mitigation approaches (including cost reserves) 
to arrive at an overall cost risk assessment and 
rationale. 


Independent Cost Assessment 
Process and Elements 



5. Overall Cost Risk 

4. Cost Assessment 

3. Cost Threats & Risks 
from all work below & 
from tech/mgt analysis 

2. Independent Tools 

- Models 

- Analogies 


1 . Analysis of 
Concept 
Study/Proposa] 


Reserve Management 
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Synthesis of Technical, Management and Cost 
Data 

Despite weaknesses and flaws found in reviews of 
each of the technical, management and cost areas, 
each area by itself may or may not indicate a poor 
chance of mission success. However, when viewed 
in the aggregate, the mission may still be 
considered to have a good chance of success even 
when significant technical flaws exist. Experience 
has shown that all development programs will 
encounter unforeseen events and problems and that 
overcoming these are linked to all the pieces. 
Predicting mission success can be tied to the level 
of maturity of the design/development in relation to 
the planned state, the degree of technical, cost and 
schedule margins for the state of the development, 
and the quality of the management approach and 
management team. 

This can be boiled down to a few simple questions: 

1 . How hard is the investigation to implement and 
how well is it understood? Is there enough 
“envelope?” What are the inherent risks? 

2. Will the overall mission design (spacecraft, 
launch vehicle, ground system, mission ops) allow 
successful implementation of mission as proposed? 
If not, are there sufficient resources (time and 
schedule) to correct identified problems? 

3. Does the proposed flight system design and 
development approach allow the mission to have a 
reasonable probability of accomplishing its 
objectives? Does it depend on advanced or new 
technology not yet demonstrated, or are the mission 
requirements within existing capabilities? Does the 
mission have sufficient resiliency in appropriate 
resources (e.g., mass, power) to accommodate 
development uncertainties? 

4. Does the schedule reveal an understanding of 
work to be done, the time it takes to do it, and is 
there a reasonable probability of launching on time? 

5. Will the management plan, organization, roles 
and responsibilities, and experience allow 
successful completion of investigation? 


6. Does the investigation have a reasonable chance 
of being accomplished within proposed cost? 

7. Does the mission team understand the risks and 
have adequate fallback plans to mitigate them, 
including risk of using new technology, to assure 
that mission can be completed as planned? 

Are the risks and the risk mitigation approaches 
adequate to give sufficient warning to ensure that 
they can be mitigated without impacting the 
mission objectives? 

If the answer to all of these questions is positive, 
then the mission is likely to succeed. For each 
negative answer, however, the level of risk must 
rise. 

Despite doing or planning to do everything 
correctly, mission success cannot be guaranteed. 
There are many intangibles that can affect mission 
success that the team may have little control over. 
Some of these intangibles are: 

Political meddling 

Customer meddling 

Adversarial relationships 

Failure of customer to meet commitments 

Poor Communications 

Geographic dispersion of project elements 

Facility problems 

These uncontrollable factors can create problems 
for an otherwise excellent mission. If there are 
adequate margins in the critical mission resources 
(cost, schedule, mass, etc.), even these problems 
might be overcome. 

Conclusions 

As can be seen, predicting mission success is as 
simple as the application of good engineering 
practices and common sense. Most flaws in 
missions with poor chances can be traced to trying 
to do too much with too little, whether it is money, 
time, technical resources, technology or 
management. Over the past 5 years, mission teams 
proposing to Discovery and Explorer 
Announcements of Opportunities have been 
learning this lesson, and the trend is for increasing 
numbers of them to achieve low risk ratings with 
each AO. This is creating a pleasant but interesting 



problem for the Associate Administrator of OSS: an 
abundance of diverse selectable missions to choose 
from. Since the discriminators are getting very 
tight, most successful proposals to NASA’s 
Discovery and Explorer programs are typically well 
into Phase A at the proposal stage. In the end, if 
missions are technically equivalent, the final 


selection may be made solely on the basis of the 
best science for the dollar cost. 
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